Workshop: Writing reproducible APA-style manuscripts in R

background-image: url("data:image/png;base64,#https://github.com/allisonhorst/stats-illustrations/raw/master/rstats-artwork/welcome_to_rstats_twitter.png")
background-position: 50% 0%
background-size: 60%
class: bottom

## Writing reproducible manuscripts in R
[**Shilaan Alzahawi**](http://shilaan.rbind.io) @ Stanford Graduate School of Business  
Artwork by [@allison_horst](https://github.com/allisonhorst/stats-illustrations)

---

### Do your data sci like it's going to need an alibi

Slides at [bit.ly/shilaan-apa](https://bit.ly/shilaan-apa)  
Artwork by [@allison_horst](https://github.com/allisonhorst/stats-illustrations)

---
background-image: url("data:image/png;base64,#https://github.com/allisonhorst/stats-illustrations/raw/master/make-your-own-stats-cartoons/ex_3.png")
background-position: 90% 50%
background-size: 38%

# Introduction 🧊🔨

`\(~~\)`  
`\(~~\)`

**Statistics** @ Ghent University  
**Organizational Behavior** @ Stanford GSB

🔎 `\(~\)` Statistical inference & hypothesis testing  
🔎 `\(~\)` Open and reproducible science  
🔎 `\(~\)` Crowdsourced & big team science

---

# Outline

???

I'll first give a very brief conceptual introduction to the what and why of reproducible manuscripts, and then we'll spend the bulk of our time on the practical aspects of writing fully reproducible manuscripts.

**What?**

📝  `\(~\)` Reproducible manuscripts

**Why?**

✅ `\(~\)` Benefits

**How?**

🛠 `\(~\)` Tutorial   
`\(~~~~~~\)` pt. 1: An introduction to **R Markdown**   
`\(~~~~~~\)` pt. 2: An introduction to **papaja**

`\(~~\)`  
Example manuscripts at [github.com/shilaan/example-manuscripts](https://github.com/shilaan/example-manuscripts)

???

On the slides, I'm sharing a link to a GitHub repo with example manuscripts that you can look at now, or look at later for future reference if you're inspired to give the things I'll discuss today a try in one of your future projects.

This repo includes a link to the slides and instructions for installing R, RStudio, and RMarkdown, and for cloning the contents of the repository to your local computer. Once you've cloned the contents of this repo, you'll be able to follow along at your own pace and take a look at three fully reproducible manuscripts generated with R Markdown.

During this talk, I'll really start with the basics, so if you've already had some prior exposure to R and RMarkdown then you may want to start with the online materials on GitHub instead.

---

# The typical workflow

When writing a scientific report, the typical workflow is to ...

???

Now you may have attended Mike’s conceptual talk on this topic last week, in which case you’ll see some overlap in the Workflow that I’m about to introduce and the personal experiences that Mike discussed last week.

1. Do your analyses (e.g., in `R`, `Python`, `SPSS`, `SAS`, `Matlab`, or `Stata`)

--
2. Copy-paste or otherwise save your graphs and results

--
3. Open a program (e.g., `Microsoft Word`) to communicate the results

4. Manually format your results and citations

### Discussion questions

What are common challenges when working in this fashion?  
What kind of problems could arise?

???

A question to keep in mind when we discuss this workflow is what kind of challenges or problems could arise, and maybe what kind of problems you've personally experienced while working in this fashion. Of course feel free to comment in the chat if you want to share your thoughts or horror stories.

---
class:center, middle

---

---

# Typical workflow challenges

- Time-consuming

???

Time related to transcribing, copy pasting, formatting references, fixing the layout of your document after you've added another Table or Figure in Word, time related to tracking down mistakes or other sources of differences when you all of a sudden get a different result...

- Error-prone (e.g., rounding or transcription errors)

???

This is something that came through very clearly in Mike Frank's talk - the prevalence of statistical reporting errors in psychological science.

- Lacks transparency; difficult to reproduce (by others **and** yourself!)

- Difficult to maintain and update (endless rewriting and reformatting...)

???

Multiple inconsistent versions of code, data, or both

- Overhead costs of different computing/software environments

- **Anything else...?**

---
background-image: url("data:image/png;base64,#https://upload.wikimedia.org/wikipedia/en/f/ff/SuccessKid.jpg")
background-position: 50% 92%
background-size: 45%

## An alternative workflow: What?

- Fuse your code and writing

- Directly embed results in your report

- Automatically reflect analytic changes in your documentation

- Update all your results, figures, and tables automatically

- Automatic formatting (including citations!)

---
background-image: url("data:image/png;base64,#https://raw.githubusercontent.com/allisonhorst/stats-illustrations/master/rstats-artwork/data_cowboy.png")
background-position: 90% 40%
background-size: 50%

## An alternative workflow: Why?

Less...

⬇️ Error-prone  
 
???
 
In the conceptual talk on reproducible manuscripts, Mike very clearly discussed the rationale behind reproducible manuscripts, with one of them being error detection and mitigation. To borrow Mike's words, a reproducible report is easier to audit, and because we avoid manual tasks like copy pasting, we reduce opportunities for error.  
 
--

⬇️ Time-consuming
 
???

Maya, Percy, and Mike also discussed this in their earlier talks in the lecture series: when you invest in reproducible research practices, there's a steep learning curve at first - but it is immensely time-saving in the long term. To again borrow Mike's words, It's easier to build on your prior work by re-using your code, and there's no need to start from scratch each time or remind yourself what you did previously. 
 
--

More...

⬆️ Dynamic     
 
--

⬆️ Reproducible

⬆️ Transparent  (for others **and** yourself)
 
???

Like Maya said in her conceptual talk on reproducible data analysis - investing in reproducible practices doesn’t just benefit other people, it benefits your future self - it’s an investment in your future sanity.

---
background-image: url("data:image/png;base64,#https://bookdown.org/yihui/rmarkdown/images/hex-rmarkdown.png")
background-position: 50% 90%
background-size: 20%

## Our weapon of choice: RMarkdown

- RMarkdown is an **authoring framework for data science**, designed for reproducibility

- The same document holds the code and the narrative surrounding the data

- Results are automatically generated from the code

- You can use a single R Markdown file to    
✓ save and execute code, and   
✓ generate high quality reports that can be shared with an audience

---

Artwork by [@allison_horst](https://github.com/allisonhorst/stats-illustrations):  
**Get your code, text, and outputs in the same (reproducible) place**

---

## Introduction to RMarkdown
--

- Create dynamic analysis documents that combine code, output (incl. figures and tables), and writing

???

A dynamic scientific report interleaves the written report with blocks of code used to conduct the analyses. The text is automatically formatted as a scientific paper in several potential styles. When the text is formatted, the code blocks are evaluated and their results are insterted in the text, or rendered as figures and tables. Dynamic document generation replaces the classical approach of using separate programs to write prose and conduct analyses, and then manually copy-pasting analysis results into the text. When revisions require major changes to the analyses, all results, figures and tables
are automatically updated. The flexibility in output formats also means that a manuscript can be rendered to PDF, to word, or even to a poster, website, or blog post.

- Can be used to  
✓ Reproduce your analyses  
✓ Collaborate and share code with others  
✓ Communicate your results with others

- Output formats include HTML, PDF, Word and...  
🤩 Slide shows ([bit.ly/shilaan-apa](https://shilaan-apa.netlify.app))    
🤩 Websites ([shilaan.rbind.io](http://shilaan.rbind.io))  
🤩 Blogs  
🤩 Books  
🤩 CVs    
🤩 Dashboards  
🤩 Interactive documents   
🤩 Conference posters  
🤩 Manuscripts

---
background-image: url("data:image/png;base64,#images/manuscript.png")
background-position: 50% 80%
background-size: contain

## Sneak peek: the power of RMarkdown
---

background-image: url("data:image/png;base64,#images/cites.png")
background-position: 50% 70%
background-size: contain

## Sneak peek: the power of RMarkdown
---

background-image: url("data:image/png;base64,#images/refs.png")
background-position: 50% 50%
background-size: contain

## Sneak peek: the power of RMarkdown
---
## Discussion question

#### Are there good reasons for **not** using **RMarkdown**?

--
<svg viewBox="0 0 512 512" style="position:relative;display:inline-block;top:.1em;fill:#035AA6;height:1.5em;" xmlns="http://www.w3.org/2000/svg">  <path d="M256 8C119.043 8 8 119.083 8 256c0 136.997 111.043 248 248 248s248-111.003 248-248C504 119.083 392.957 8 256 8zm0 448c-110.532 0-200-89.431-200-200 0-110.495 89.472-200 200-200 110.491 0 200 89.471 200 200 0 110.53-89.431 200-200 200zm107.244-255.2c0 67.052-72.421 68.084-72.421 92.863V300c0 6.627-5.373 12-12 12h-45.647c-6.627 0-12-5.373-12-12v-8.659c0-35.745 27.1-50.034 47.579-61.516 17.561-9.845 28.324-16.541 28.324-29.579 0-17.246-21.999-28.693-39.784-28.693-23.189 0-33.894 10.977-48.942 29.969-4.057 5.12-11.46 6.071-16.666 2.124l-27.824-21.098c-5.107-3.872-6.251-11.066-2.644-16.363C184.846 131.491 214.94 112 261.794 112c49.071 0 101.45 38.304 101.45 88.8zM298 368c0 23.159-18.841 42-42 42s-42-18.841-42-42 18.841-42 42-42 42 18.841 42 42z"></path></svg> `\(~~~\)` Anything else?

---

# Part 1: RMarkdown

---

# Getting started with RMarkdown

- Install [`R`](https://cran.r-project.org/mirrors.html)
- Install [`RStudio`](https://www.rstudio.com/products/rstudio/download/)
- Install the `RMarkdown` package  
- Install `\(\LaTeX\)` (e.g., `TinyTex`)

```r
install.packages("rmarkdown")
install.packages("tinytex") # for generating PDF output
tinytex::install_tinytex()  # install TinyTeX
```

---

## Opening a new R Markdown

- Create a new R Markdown document from the menu  
`File -> New File -> R Markdown`

---

## Notebook interface

- Allows for direct interaction with R (execute code and display results inline)
- Makes it easy to test and iterate
- Produces a reproducible document with publication-quality output

---

## Three types of content
- YAML meta-data / frontmatter (between `---` and `---`)
- Text with Markdown formatting
- R code

---

# Metadata

---
background-image: url("data:image/png;base64,#https://cwextensions.com/images/logo-someta.png")
background-position: 92% 7%
background-size: 10%

# YAML metadata

The YAML header contains basic metadata and rendering instructions

```yaml
---
title: My R Markdown Report
author: Shilaan Alzahawi
output: pdf_document
date: "2021-11-04"
---
```

The date will be **dynamically updated** every time we knit the report,  
with the help of the following line of code (more on **in-line code** later):

---

# Preview an RMarkdown

---

# Rendering a document

✓ ![knit](data:image/png;base64,#images/knit.png)  
✓ Windows/Linux: `Control + Shift + K`   
✓ OS X: `Command + Shift + K`

---

Artwork by [@allison_horst](https://github.com/allisonhorst/stats-illustrations):  
**Become an RMarkdown knitting wizard**

---

## Output formats

![](data:image/png;base64,#https://d33wubrfki0l68.cloudfront.net/ece57b678854545e6602a23daede51ad72da2170/21cca/lesson-images/outputs-1-word.png)

---

## Output formats

![](data:image/png;base64,#https://d33wubrfki0l68.cloudfront.net/ebcf2beeb67cc21693d73b8708d5af0fa9769f57/de9a9/lesson-images/outputs-2-pdf.png)

---

## What's happening behind the scenes?

![A diagram illustrating how an R Markdown document is converted to the final output document](data:image/png;base64,#https://bookdown.org/yihui/rmarkdown-cookbook/images/workflow.png)

☞ The code within the `.Rmd` file is executed and converted into an `.md` file;  
☞ The `.md` file is converted to the output format specified in the metadata

---

## What's happening behind the scenes?

Knitting an `RMarkdown` file...

1. Starts a new R session  
✓ No packages or objects loaded

2. Sets your working directory to the location of the `RMarkdown` file

3. Executes all code chunks from top to bottom

### ⚠️ **Make sure to load all R packages you use!**

<img src="data:image/png;base64,#https://github.com/allisonhorst/stats-illustrations/raw/master/make-your-own-stats-cartoons/ex_4.png" width="45%" style="display: block; margin: auto;" />
Artwork by [@allison_horst](https://github.com/allisonhorst/stats-illustrations)

---

# Code

---

## Two types of code in RMarkdown

1. A code chunk, surrounded by three backticks and `{r}`
2. An inline code expression, surrounded by one backtick and `r`

---

## Code chunks

"*Code chunks are the beating heart of our R Markdown.*" 
[Xie, Dervieux, Riederer 2021](https://bookdown.org/yihui/rmarkdown-cookbook/rmarkdown-anatomy.html)

```r
summary(Orange)
```

```
##  Tree       age         circumference  
##  3:7   Min.   : 118.0   Min.   : 30.0  
##  1:7   1st Qu.: 484.0   1st Qu.: 65.5  
##  5:7   Median :1004.0   Median :115.0  
##  2:7   Mean   : 922.1   Mean   :115.9  
##  4:7   3rd Qu.:1372.0   3rd Qu.:161.5  
##        Max.   :1582.0   Max.   :214.0
```

### Inserting a code chunk

--
✓ Windows/Linux: `Control + Alt + I`

--
✓ OS X: `Command + Option + I`

--
✓ Enclosing code with three backticks and `{r}`

--
✓ ![](data:image/png;base64,#https://d33wubrfki0l68.cloudfront.net/b8b19518e688e3ca1390e0a1588916f04908d33f/8a4dc/images/notebook-insert-chunk.png)
---

## Inserting code chunks

---

## Chunk anatomy

---

## Naming your code chunks

It's recommended to name your chunks. This allows you to quickly navigate code, automatically name figures, and troubleshoot errors.

---

## Chunk options

Control a chunk's behavior by passing additional, comma-separated arguments

✓ `echo = TRUE` show code and output (*default*)

✓ `echo = FALSE` show output only (hide code)

✓ `include = FALSE` do not show output (run code)

✓ `eval = FALSE` show code (do not run; no output)

✓ `warning = FALSE` removes warning messages

✓ `error = FALSE` removes error messages

✓ `message = FALSE` removes all messages

```r
summary(Orange)
```

--
**Bonus question:** What chunk option did I set here?

---

## Chunk options

Credit for all GIFs goes to [Shannon Pileggi](https://www.pipinghotdata.com/posts/2020-09-07-introducing-the-rstudio-ide-and-r-markdown/#rmd)

---
## Chunk execution

`Ctrl + Enter` or `Command + Enter` or press ![](data:image/png;base64,#https://www.pipinghotdata.com/posts/2020-09-07-introducing-the-rstudio-ide-and-r-markdown/img/run-chunk.PNG)

---

## In-line code

To insert in-line code, wrap your code in a single backtick. RMarkdown will always
- display the results of inline code, but not the code  
- apply relevant text formatting to the results

**R Markdown document**   
<img src="data:image/png;base64,#images/inline.png" width="1867" />

--
**Knitted HTML document**  
<img src="data:image/png;base64,#images/inline-knitted.png" width="1667" />

---

# Text

---

# Markdown formatting basics

![](data:image/png;base64,#https://d33wubrfki0l68.cloudfront.net/59f29676ef5e4d74685e14f801bbc10c2dbd3cef/c0688/lesson-images/markdown-1-markup.png)

---

![](data:image/png;base64,#images/syntax-becomes.png)

For more formatting options, see the [R Markdown Reference guide](https://www.rstudio.com/wp-content/uploads/2015/03/rmarkdown-reference.pdf?_ga=2.157796986.1542626288.1625161001-1806201684.1624641897)

---

## Tables